<!DOCTYPE html>
<html lang="zh">
    <head>
        <meta charset="utf-8">
        <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
        <title>cs229 第五节 | Dirac Lee</title><meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="robots" content="noodp" />
<meta name="Description" content="cs229 第五节"><link rel="prev" href="https://diraclee.gitee.io/2020/07/cs229_lecture4/" /><link rel="next" href="https://diraclee.gitee.io/2020/07/leetcode0004/" /><link rel="canonical" href="https://diraclee.gitee.io/2020/07/cs229_lecture5/" />
<link rel="shortcut icon" type="image/x-icon" href="/favicon.ico" />
<link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon.png">
<link rel="icon" type="image/png" sizes="32x32" href="/favicon-32x32.png">
<link rel="icon" type="image/png" sizes="16x16" href="/favicon-16x16.png">
<link rel="manifest" href="/site.webmanifest">
<link rel="mask-icon" href="/safari-pinned-tab.svg" color="#5bbad5">
<meta name="msapplication-TileColor" content="#da532c">
<meta name="theme-color" content="#ffffff"><meta property="og:title" content="cs229 第五节" />
<meta property="og:description" content="cs229 第五节" />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://diraclee.gitee.io/2020/07/cs229_lecture5/" />
<meta property="article:published_time" content="2020-07-23T00:00:00+00:00" />
<meta property="article:modified_time" content="2020-07-23T00:00:00+00:00" />
<script type="application/ld+json">
    {
        "@context": "http://schema.org",
        "@type": "BlogPosting",
        "headline": "cs229 第五节",
        "mainEntityOfPage": {
            "@type": "WebPage",
            "@id": "https:\/\/diraclee.gitee.io\/2020\/07\/cs229_lecture5\/"
        },"image": {
                "@type": "ImageObject",
                "url": "https:\/\/diraclee.gitee.io\/logo.png",
                "width":  800 ,
                "height":  600 
            },"genre": "posts","keywords": "机器学习, cs229","wordcount":  1356 ,
        "url": "https:\/\/diraclee.gitee.io\/2020\/07\/cs229_lecture5\/","datePublished": "2020-07-23T00:00:00\x2b00:00","dateModified": "2020-07-23T00:00:00\x2b00:00","license": "This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.","publisher": {
                "@type": "Organization",
                "name": "xxxx",
                "logo": {
                "@type": "ImageObject",
                "url": "https:\/\/diraclee.gitee.io\/logo.png",
                "width":  127 ,
                "height":  40 
                }
            },"description": "cs229 第五节"
    }
    </script><link rel="stylesheet" href="/css/style.min.css"><link rel="stylesheet" href="/css/lib/fontawesome-free/all.min.css"><link rel="stylesheet" href="/css/lib/animate/animate.min.css"></head>
    <body><script>
            window.isDark = (window.localStorage && window.localStorage.getItem('theme')) === 'dark';
            window.isDark && document.body.classList.add('dark-theme');
        </script><div class="wrapper"><nav class="navbar">
    <div class="navbar-container">
        <div class="navbar-header animated bounceIn">
            <a href="https://diraclee.gitee.io">Dirac Lee</a>
        </div>
        <div class="navbar-menu"><a class="menu-item" href="https://diraclee.gitee.io/posts" title="">文章</a><a class="menu-item" href="https://diraclee.gitee.io/tags" title="">标签</a><a class="menu-item" href="https://diraclee.gitee.io/categories" title="">分类</a><a class="menu-item" href="https://diraclee.gitee.io/about" title="">关于</a><a class="menu-item" href="https://diraclee.gitee.io" title="English"><i class="fas fa-language fa-fw"></i></a><a href="javascript:void(0);" class="theme-switch"><i class="fas fa-adjust fa-rotate-180 fa-fw" title="切换主题"></i></a>
        </div>
    </div>
</nav><nav class="navbar-mobile">
    <div class="navbar-container">
        <div class="navbar-header">
            <div class="navbar-header-title animated bounceIn">
                <a href="https://diraclee.gitee.io">Dirac Lee</a>
            </div>
            <div class="menu-toggle" id="menu-toggle">
                <span></span><span></span><span></span>
            </div>
        </div>
        <div class="navbar-menu" id="mobile-menu"><a class="menu-item" href="https://diraclee.gitee.io/posts" title="">文章</a><a class="menu-item" href="https://diraclee.gitee.io/tags" title="">标签</a><a class="menu-item" href="https://diraclee.gitee.io/categories" title="">分类</a><a class="menu-item" href="https://diraclee.gitee.io/about" title="">关于</a><a class="menu-item" href="https://diraclee.gitee.io" title="English"></a><a href="javascript:void(0);" class="theme-switch"><i class="fas fa-adjust fa-rotate-180 fa-fw" title="切换主题"></i></a>
        </div>
    </div>
</nav><main class="main">
                <div class="container"><article class="page"><h1 class="post-title animated flipInX">cs229 第五节</h1><div class="post-meta">
            <div class="post-meta-main"><a class="author" href="https://diraclee.gitee.io" rel="author" target="_blank">
                    <i class="fas fa-user-circle fa-fw"></i>Dirac Lee
                </a>&nbsp;<span class="post-category">收录于&nbsp;<i class="far fa-folder fa-fw"></i><a href="https://diraclee.gitee.io/categories/%E5%AD%A6%E4%B9%A0%E7%AC%94%E8%AE%B0/">学习笔记</a>&nbsp;</span></div>
            <div class="post-meta-other"><i class="far fa-calendar-alt fa-fw"></i><time datetime=2020-07-23>2020-07-23</time>&nbsp;
                <i class="fas fa-pencil-alt fa-fw"></i>约 1356 字&nbsp;
                <i class="far fa-clock fa-fw"></i>预计阅读 3 分钟&nbsp;</div>
        </div><div class="post-content"><p>概率论</p>
<ul>
<li>术语</li>
<li>一元随机变量</li>
<li>二元随机变量</li>
<li>贝叶斯理论</li>
<li>多元高斯分布</li>
</ul>
<a class="post-dummy-target" id="术语"></a><h2>术语</h2>
<a class="post-dummy-target" id="样本空间-omega"></a><h3>样本空间 $\Omega$</h3>
<ul>
<li>例如抛硬币抛两次的结果 $\Omega = \lbrace HH, HT, TH, TT \rbrace$</li>
</ul>
<a class="post-dummy-target" id="事件-a-in-omega"></a><h3>事件 $A \in \Omega$</h3>
<ul>
<li>例如 $A =$ 第一次抛正面向上</li>
</ul>
<a class="post-dummy-target" id="事件空间-mathscr-f"></a><h3>事件空间 $\mathscr F$</h3>
<ul>
<li>例如 $\mathscr F_A = \lbrace HH, HT \rbrace \in \Omega$</li>
</ul>
<a class="post-dummy-target" id="概率评估函数-p-mathscr-f-rightarrow-r"></a><h3>概率评估函数 $P: \mathscr F \rightarrow \R$</h3>
<ul>
<li>对 $\forall A \in \mathscr F$, $P(A) \ge 0$</li>
<li>$P(\Omega) = 1$</li>
<li>若 $A_1$, $A_2$, &hellip; 为不相交事件集( 即 $A_i \cap A_j = \emptyset$ )，则<br>
$$ P(\cup_i A_i) = \sum_i P(A_i) $$</li>
</ul>
<a class="post-dummy-target" id="条件概率"></a><h3>条件概率</h3>
<ul>
<li>若事件B发生的概率 $P(B) \ne 0$，则<br>
$$
P(A | B) = \frac {P(A \cap B)} { P(B)}
$$</li>
</ul>
<a class="post-dummy-target" id="独立"></a><h3>独立</h3>
<ul>
<li>若事件A与B相互独立，则<br>
$$
P(A \cap B) = P(A) P(B)
$$</li>
</ul>
<a class="post-dummy-target" id="一元随机变量"></a><h2>一元随机变量</h2>
<a class="post-dummy-target" id="随机变量-rv"></a><h3>随机变量 (RV)</h3>
<ul>
<li>$\omega_i$ 是一次实验结果</li>
<li>$\Omega_i$ 是所有实验结果组成的集合</li>
<li>$X(\omega_i)$ 是这次实验结果中我们所关心的一个数字</li>
<li>$X(\Omega_i)$ 是这类实验中该数字的所有可能值</li>
<li>事件&quot;抛10次硬币的结果为 $\lbrace HHHTHTTHTT \rbrace$&rdquo; 为 $\omega_0$，<br>
则该事件的表示&quot;正面次数&rdquo; 的随机变量 $X(\omega_0) = 5$</li>
<li>记 $Val(X) = X(\Omega)$</li>
</ul>
<a class="post-dummy-target" id="离散随机变量"></a><h3>离散随机变量</h3>
<p>离散随机变量 $Val(X)$ 是可计数的</p>
<p>在某个特定点的概率 $P(X = k) = P({\omega | X(\omega) = k})$</p>
<a class="post-dummy-target" id="概率质量函数-pmf"></a><h3>概率质量函数 (PMF)</h3>
<p>离散随机变量的概率质量函数 $P_X: Val(X) \rightarrow [0, 1]$</p>
<ul>
<li>$P_X(x) = P(X = x)$</li>
<li>$\begin{aligned} \sum_{x \in Val(X)} P_X (x) = P(\Omega) = 1 \end{aligned}$</li>
</ul>
<a class="post-dummy-target" id="连续随机变量"></a><h3>连续随机变量</h3>
<p>连续随机变量 $Val(X)$ 是不可计数的</p>
<p>在某一区间范围内的概率 $P(a \le X \le b) = P(\lbrace \omega | a \le X(\omega) \le b  \rbrace)$</p>
<a class="post-dummy-target" id="概率密度函数-pdf"></a><h3>概率密度函数 (PDF)</h3>
<p>$f_X: \R \rightarrow \R$</p>
<ul>
<li>$f_X(x) = \frac d {dx} F_X(x)$</li>
<li>$f_X(x) \ne P(X = x)$</li>
<li>$ \int_{-\infin}^{\infin}   \underbrace{f_X(x)dx}_{\text{P(x ≤ X ≤ x + dx)}} = 1 $</li>
</ul>
<a class="post-dummy-target" id="累计分布函数-cdf"></a><h3>累计分布函数 (CDF)</h3>
<p>$F_X: \R \rightarrow [0, 1]$</p>
<p>$F_X = P(X \le x) = P(\lbrace \omega | X(\omega) \le x  \rbrace)$</p>
<a class="post-dummy-target" id="均值"></a><h3>均值</h3>
<p>$g: \R \rightarrow \R$</p>
<p>若 $X$ 为离散随机变量，其 PMF 为 $P_X$，则 $$E[g(x)] = \sum_{x \in Val(X)} g(x) P_X(x)$$</p>
<p>例如</p>
<p>$X \sim Ber(\phi)$，
$P(X) = 
\begin{cases}
\phi &amp; x = 1 \\ 
1 - \phi &amp; x = 0
\end{cases}$，
$g(x) = x$，则</p>
<p>$$
E[X] = \sum_{x \in \lbrace 0, 1 \rbrace} g(x) P_X(x) = 1 \times \phi + 0 \times (1 - \phi) = \phi
$$</p>
<p>再如</p>
<p>$X \sim Poss(\lambda)$，
$P_X(x) = \frac {e^{-\lambda} \lambda^x} {x!}$，
$g(x) = x$，则</p>
<p>$$
\begin{aligned}
E[x] 
&amp;= \sum_{x = 0}^\infin g(x) P_X(x) 
= \sum_{x = 0}^\infin \frac {x e^{-\lambda} \lambda^x} {x!} 
= \sum_{x = 1}^\infin \frac { e^{-\lambda} \lambda^x} {(x-1)!}  \\ 
&amp;= \lambda e^{-\lambda} \underbrace{ \sum_{x = 1}^\infin \frac {\lambda^{x - 1}} {(x - 1)!}}_{\text{exp(λ)}}
= \lambda
\end{aligned}
$$</p>
<a class="post-dummy-target" id="二元随机变量"></a><h2>二元随机变量</h2>
<a class="post-dummy-target" id="cdf"></a><h3>CDF</h3>
<p>$$F_{XY} (x, y) = P(X \le x, Y \le y)$$</p>
<a class="post-dummy-target" id="pmf"></a><h3>PMF</h3>
<p>$$P_{XY} (x, y) = P(X = x, Y = y)$$</p>
<a class="post-dummy-target" id="pdf"></a><h3>PDF</h3>
<p>$$f_{XY} (x, y) = \frac {\partial^2} {\partial x \partial y} F_{XY} (x, y)$$</p>
<a class="post-dummy-target" id="边缘-pmf"></a><h3>边缘 PMF</h3>
<p>$$P_X (x) = \sum_y P_{XY} (x, y)$$</p>
<a class="post-dummy-target" id="边缘-pdf"></a><h3>边缘 PDF</h3>
<p>$$f_X(x) = \int_{-\infin}^{\infin} f_{XY} (x, y)$$</p>
<a class="post-dummy-target" id="独立-1"></a><h3>独立</h3>
<p>两个随机变量 X 和 Y 满足
$$
P_{XY}(x, y) = P_X(x) P_Y(y)
$$</p>
<p>或
$$
P_{Y|X} (x, y) = P_Y(y)
$$</p>
<p>则称 X 与 Y 是独立的</p>
<p>至于连续随机变量，只需将 P 换做 f 即可</p>
<a class="post-dummy-target" id="均值-1"></a><h3>均值</h3>
<p>X 和 Y 为两个连续随机变量</p>
<p>$g: \R^2 \rightarrow \R$ 为关于 X 和 Y 的函数</p>
<p>则 
$$
E[g(X, Y)] = \int_{x \in Val(X)} \int_{y \in Val(Y)} g(x, y) f_{XY} (x, y) dx dy
$$</p>
<a class="post-dummy-target" id="协方差"></a><h3>协方差</h3>
<p>定性描述：改变一个随机变量能够多大程度上影响另一个随机变量</p>
<p>$$
Cov[X, Y] = E[(x - E_X)(y - E_Y)]
$$</p>
<p>若 X 与 Y 独立，则</p>
<p>$E[XY] = E_X E_Y$</p>
<p>$Cov[X, Y] = 0$</p>
<p>$Var[X + Y] = (E[X + Y])^2 - E[(X + Y)^2]$</p>
<p>$Var[X + Y] = Var[X] + Var[Y] + 2 Cov[X, Y]$</p>
<a class="post-dummy-target" id="贝叶斯理论"></a><h2>贝叶斯理论</h2>
<p>已知 y 发生的前提下 x 发生的条件概率 $P(x | y)$，
那么 x 发生的前提下 y 发生的条件概率</p>
<p>$$
P(y | x) = \frac {P(x | y) P(y)} {P(x)}
$$</p>
<p>其中 $P(x) = \sum_{y_i \in Val(y)}  P(x | y_i) P(y_i)$</p>
<p>当 X 与 Y 连续时</p>
<p>$$
f(y | x) = \frac {f(x | y) f(y)} {f(x)}
$$</p>
<p>其中 $f(x) = \int_{y_i \in Val(y)}  f(x | y_i) f(y_i) d y_i$</p>
<p>例如</p>
<ul>
<li>A箱中有100个金币</li>
<li>B箱中有50个金币，50个银币</li>
</ul>
<p>若随机选择一个箱子从中取出一枚硬币，发现它是金币
那么选择的是A箱的概率</p>
<p>$$
\begin{aligned}
P(x | y) 
&amp;= \frac {P(y | x) P(x)} {P(y)}  \\ 
&amp;= \frac {P(y | x) P(x)} {P(x) P(y | x) + P(\overline x) P(y | \overline x)}  \\ 
&amp;= \frac {1 \times \frac 1 2} {\frac 1 2 \times 1 + \frac 1 2 \times \frac 1 2}  \\ 
&amp;= \frac 2 3
\end{aligned}
$$</p>
<p>其中，$X$: 抽中 A 箱，$Y$: 抽中金币</p>
<a class="post-dummy-target" id="多元高斯分布"></a><h2>多元高斯分布</h2>
<p>$x = \begin{bmatrix} x_1 \\  x_2 \\ \vdots \\ x_n \end{bmatrix}$，每个元素都服从高斯分布，即 $x_i \sim \mathcal N(\mu_i, \sigma_i)$</p>
<p>均值向量 $\mu = \begin{bmatrix} \mu_1 \\  \mu_2 \\ \vdots \\ \mu_n \end{bmatrix}$</p>
<p>协方差矩阵 $\Sigma = \begin{bmatrix} 
\sigma^2_{x_1} &amp; cov[x_1, x_2] &amp; \dots &amp; cov[x_1, x_n] \\ 
\\ 
cov[x_2, x_1] &amp; \sigma^2_{x_2} &amp; \dots &amp; cov[x_2, x_n] \\ 
\\ 
\vdots &amp; \vdots &amp; \ddots &amp; \vdots \\ 
\\ 
cov[x_n, x_1] &amp; cov[x_n, x_2] &amp; \dots &amp; \sigma^2_{x_n}
\end{bmatrix}$ 为对称矩阵</p>
<p>则概率质量函数</p>
<p>$$
P(x; \mu, \Sigma) = \frac 1 {(2\pi)^{\frac n 2} |\Sigma|^{\frac 1 2 }} \exp[- \frac 1 2 (x - \mu)^T \Sigma^{-1} (x - \mu)]
$$</p>
<a class="post-dummy-target" id="参考"></a><h2>参考</h2>
<p><a href="https://katex.org/docs/supported.html" target="_blank">$\KaTeX$ 文档</a></p></div><div class="post-footer" id="post-footer">
    <div class="post-info">
        <div class="post-info-line">
            <div class="post-info-mod">
                <span>本文于 2020-07-23 更新</span>
            </div>
            <div class="post-info-license"></div>
        </div>
        <div class="post-info-line">
            <div class="post-info-md"></div>
            <div class="post-info-share"><span><a href="//www.linkedin.com/shareArticle?url=https%3a%2f%2fdiraclee.gitee.io%2f2020%2f07%2fcs229_lecture5%2f&amp;title=cs229%20%e7%ac%ac%e4%ba%94%e8%8a%82" target="_blank" title="分享到 LinkedIn">
            <i class="fab fa-linkedin fa-fw"></i>
        </a><a href="//service.weibo.com/share/share.php?url=https%3a%2f%2fdiraclee.gitee.io%2f2020%2f07%2fcs229_lecture5%2f&amp;appkey=&amp;title=cs229%20%e7%ac%ac%e4%ba%94%e8%8a%82" target="_blank" title="分享到 Weibo">
            <i class="fab fa-weibo fa-fw"></i>
        </a></span></div>
        </div>
    </div>

    <div class="post-info-more">
        <section><span class="tag">
                        <a href="https://diraclee.gitee.io/tags/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0/"><i class="fas fa-tag fa-fw"></i>&nbsp;机器学习</a>&nbsp;
                    </span><span class="tag">
                        <a href="https://diraclee.gitee.io/tags/cs229/"><i class="fas fa-tag fa-fw"></i>&nbsp;cs229</a>&nbsp;
                    </span></section>
        <section>
            <span><a href="javascript:window.history.back();">返回</a></span>&nbsp;|&nbsp;<span><a href="https://diraclee.gitee.io">主页</a></span>
        </section>
    </div>

    <div class="post-nav"><a href="https://diraclee.gitee.io/2020/07/cs229_lecture4/" class="prev" rel="prev" title="cs229 第四节"><i class="fas fa-angle-left fa-fw"></i>cs229 第四节</a>
            <a href="https://diraclee.gitee.io/2020/07/leetcode0004/" class="next" rel="next" title="LeetCode0004 寻找两个正序数组的中位数">LeetCode0004 寻找两个正序数组的中位数<i class="fas fa-angle-right fa-fw"></i></a></div>
</div><div class="post-comment"></div>
    </article></div>
            </main><footer class="footer">
    <div class="copyright"><div class="copyright-line">由 <a href="https://gohugo.io/" target="_blank" rel="external nofollow noopener noreffer">Hugo</a> 强力驱动 | 主题 - <a href="https://github.com/dillonzq/LoveIt" target="_blank" rel="external nofollow noopener noreffer">LoveIt<i class="far fa-heart fa-fw"></i></a>
        </div>

        <div class="copyright-line"><i class="far fa-copyright fa-fw"></i><span itemprop="copyrightYear">2020</span><span class="author" itemprop="copyrightHolder">&nbsp;<a href="https://diraclee.gitee.io" target="_blank">Dirac Lee</a></span>&nbsp;|&nbsp;<span class="license"><a rel="license external nofollow noopener noreffer" href="https://creativecommons.org/licenses/by-nc/4.0/" target="_blank">CC BY-NC 4.0</a></span></div>
    </div>
</footer></div><a href="#" class="dynamic-to-top" id="dynamic-to-top" data-scroll>
            <span>&nbsp;</span>
        </a><script src="/js/lib/jquery/jquery.slim.min.js"></script><script src="/js/lib/lazysizes/lazysizes.min.js"></script><script src="/js/lib/smooth-scroll/smooth-scroll.polyfills.min.js"></script><script>window.scroll = new SmoothScroll('[data-scroll]', {speed: 300, speedAsDuration: true});</script><link rel="stylesheet" href="/css/lib/katex/katex.min.css"><script src="/js/lib/katex/katex.min.js"></script><script defer src="/js/lib/katex/auto-render.min.js"></script><link rel="stylesheet" href="/css/lib/katex/copy-tex.min.css"><script defer src="/js/lib/katex/copy-tex.min.js"></script><script defer src="/js/lib/katex/mhchem.min.js"></script><script>
        document.addEventListener("DOMContentLoaded", function () {
            renderMathInElement(document.body, {
                delimiters: [
                    { left: "$$", right: "$$", display: true },
                    { left: "\\(", right: "\\)", display: false },
                    { left: "\\[", right: "\\]", display: true },{ left: "$", right: "$", display: false },]
            });
        });
    </script><script src="/js/blog.min.js"></script></body>
</html>